Passive/Active Load Balancing with Informed Node Placement in DHTs
نویسندگان
چکیده
Distributed key/value stores are a basic building block for large-scale Internet services. Support for range queries introduces new challenges to load balancing since both the key and workload distribution can be non-uniform. We build on previous work based on the power of choice to present algorithms suitable for active and passive load balancing that adapt to both the key and workload distribution. The algorithms are evaluated in a simulated environment, focusing on the impact of load balancing on scalability under normal conditions and in an overloaded system.
منابع مشابه
Balancing locality and randomness in DHTs
Embedding locations in DHT node IDs makes locality explicit and, thereby, enables engineering of the trade-off between careful placement and randomized load balancing. This paper discusses hierarchical, topology-exposed DHTs and their benefits for content locality, and administrative control and routing locality. We thank the members and companies of the PDL Consortium (including EMC, Hewlett-P...
متن کاملBalancing Locality and Randomness in DHTs (CMU-CS-03-203)
Embedding locations in DHT node IDs makes locality explicit and, thereby, enables engineering of the trade-off between careful placement and randomized load balancing. This paper discusses hierarchical, topology-exposed DHTs and their benefits for content locality, and administrative control and routing locality. We thank the members and companies of the PDL Consortium (including EMC, Hewlett-P...
متن کاملAdaptive Dynamic Data Placement Algorithm for Hadoop in Heterogeneous Environments
Hadoop MapReduce framework is an important distributed processing model for large-scale data intensive applications. The current Hadoop and the existing Hadoop distributed file system’s rack-aware data placement strategy in MapReduce in the homogeneous Hadoop cluster assume that each node in a cluster has the same computing capacity and a same workload is assigned to each node. Default Hadoop d...
متن کاملLocality-Aware Randomized Load-Balancing Algorithms in Structured Peer-to-Peer Networks
This paper presents a locality-aware randomized load-balancing algorithm to deal with both the proximity and network churn at the same time. A factor of randomness in the probing of lightly loaded nodes in a range of proximity is introduced. Further the efficiency by allowing the probing of multiple candidates (d-way) at a time is improved. Simulation results show the superiority of the localit...
متن کاملComparison of Load Balancing Algorithms for Structured Peer-to-Peer Systems
Among other things, Peer-to-Peer (P2P) systems are very useful for managing large amounts of widely distributed data. Distributed Hash Tables (DHT) offer a highly scalable and self-organizing approach for efficient and persistent distribution and retrieval of data. However the scalability and performance of DHTs is strongly based on an equal distribution of data across participating nodes. Beca...
متن کامل